Validation Study of the Use of PF Ratio for Patients with Acute Hypoxaemic Respiratory Failure Admitted to Australian and New Zealand Intensive Care Units

Statistical Analysis

Author

Dr Benjamin Moran, MBBS, BMedSci (Hons), MMedStats, FCICM, FANZCA

Published

February 16, 2025

1 Introduction

This is an explanation of the statistical analysis for the study validating the use of PaO2:FiO2 ratio in patients admitted to ICU with acute hypoxaemic respiratory failure.

2 Methods

This is a retrospective study using data from the Australian and New Zealand Intensive Care Society (ANZICS) adult ICU patient database (APD). This manuscript has been prepared and reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement.

2.1 Patient Population

We included all patients in the ANZICS APD from 01/01/2018 to 31/12/2022.

2.2 Aims and Objectives

  1. Determine the association between PaO2:FiO2 ratio and hospital mortality.
  2. Determine the association between PaO2:FiO2 ratio and ICU mortality.
  3. Determine the association between PaO2:FiO2 ratio and mortality at other timepoints (28 day, 6 months, 12 months).
  4. Validate the PaO2:FiO2 ratio as a diagnostic tool for prediction of hospital mortality, and mortality at other times (ICU, 28-day, 6 months, 12 months).
  5. Determine the validity of the PaO2:FiO2 ratio in predicting hospital mortality in pre-specified subgroups (invasive ventilation, ventilatory support, age, sex, admission diagnoses, frailty categories, presence of treatment limitations).

2.3 Statistical Analysis

We summarized baseline ICU and patient-level characteristics and unadjusted outcomes using standard descriptive statistics. For categorical data, we used counts and percentages, and for continuous data we used mean ± standard deviation (SD) or median (interquartile range, IQR) as appropriate depending on the distribution of data.

2.3.1 Association Between PaO2:FiO2 Ratio and Hospital Mortality

The unadjusted association between PaO2:FiO2 ratio and hospital mortality was evaluated visually as a continuous, non-linear variable using restricted cubic splines with 4 knots. Specific unadjusted hospital mortality values for standard PaO2:FiO2 ratio values were also calculated through interrogation of the spline curve.

2.3.2 Validation of PaO2:FiO2 ratio in Predicting Hospital Mortality

To determine the validitiy of the PaO2:FiO2 ratio, the area under the receiver operator characteristic curve (AUCROC) was calculated. 95% confidence intervals around the AUC were calculated using 1,000 bootstrap samples.

To determine the optimal cut-off of PaO2:FiO2 ratio in predicting hospital mortality, receiver operator characteristic curves (ROC) were generated. A cut-off level representing the highest sum of sensitivity and specificity hased on each patients PaO2:FiO2 ratio was calculated using the Youden method1. In this method, the sensitivity and specificity was calculated over a range of PaO2:FiO2 ratios. For each value, the Youden’s J index was calculated by using the following formula (Youden = Sensitivity + Specificity - 1). The value that corresponds to the highest Youden index was identified as the optimal cut-off, reflecting the highest sum of sensitivity and specifity. We also calculated the sensitivity, specificity, negative predictive value and positive predictive value for this cut-off.

2.3.3 Subgroup Analysis

Patients were analysed for validation of the PaO2:FiO2 ratio in prediction of hospital mortality in the following subgroups: receiving invasive ventilation during the index ICU admission, receiving of other respiratory support (eg non-invasive ventilation, extracorporeal membrane oxygenation) during the ICU admission, sex, age categories, admission diagnoses (medical, cardiac surgery, neurosurgery/trauma, sepsis, post-operative), frailty category (fit/well, mild, moderate, severe frailty) and the presence of treatment limitation status on ICU admission. The same methodology as above was employed. Comparison of each subgroup (ie IPPV vs Non-IPPV) was performed using DeLong’s method2.

As there were >800,000 patients in the dataset, a 2-sided p-value of 0.001 was used for statistical significance. Given that there is an increased risk of Type-1 error with multiple testing, the results of the secondary objectives should be viewed as exploratory. Hence, no adjustment for multiplicity was used. Only patients with complete data for all covariates were included in the analysis. Statistical analyses were performed using R Version 4.3.1 (R Core Team, R Foundation for Statistical Computing, Vienna, Austria) and RStudio Version 2023.12.1 (Posit Software, PBC, Boston, MA). Packages used for analysis included tidytable3, tidyverse4, data.table5, gtsummary6, gt7, cutpointr8 and pROC9.

3 Results

Of 662,612 patients admitted to 211 ICUs during the study period, 337,851 (51.0%) patients had acute hypoaxemic respiratory failure. Of this cohort, 181,499 (27.4%) had mild AHRF, 128,227 (19.4%) had moderate AHRF and 28,125 (4.2%) had severe AHRF.

3.1 Patient Demographics

Below are the demographic tables. This table has the 4 AHRF categories (none, mild, moderate, severe) to look at the breakdown of patients within each AHRF category.

Characteristic Overall1
Acute Hypoxaemic Respiratory Failure Category
None (PF >300)1 Mild (PF 200-300)1 Moderate (PF 100-200)1 Severe (PF < 100)1
Number of Patients 662,612 324,761 181,499 128,227 28,125
Median Age in Years (IQR) 66 (53-75) 65 (50-75) 68 (56-76) 67 (55-75) 65 (52-75)
Age Category, Years




    <44 99,985 (15%) 60,314 (19%) 19,751 (11%) 15,515 (12%) 4,405 (16%)
    >84 47,408 (7.2%) 24,404 (7.5%) 13,341 (7.4%) 7,985 (6.2%) 1,678 (6.0%)
    45-64 198,888 (30%) 94,398 (29%) 54,892 (30%) 40,546 (32%) 9,052 (32%)
    65-84 315,855 (48%) 145,422 (45%) 93,376 (51%) 64,083 (50%) 12,974 (46%)
Gender




    Female 275,502 (42%) 146,046 (45%) 70,722 (39%) 48,420 (38%) 10,314 (37%)
    Male 386,572 (58%) 178,437 (55%) 110,613 (61%) 79,734 (62%) 17,788 (63%)
    Intersex/Indeterminate 378 (<0.1%) 200 (<0.1%) 108 (<0.1%) 52 (<0.1%) 18 (<0.1%)
    Unknown 160 (<0.1%) 78 (<0.1%) 56 (<0.1%) 21 (<0.1%) 5 (<0.1%)
Median APACHE II Score (IQR) 15 (11-20) 13 (10-18) 15 (12-20) 18 (14-24) 23 (18-30)
Median APACHE III Score (IQR) 50 (38-66) 45 (34-59) 51 (39-66) 59 (46-77) 75 (57-99)
Median ANZROD (IQR) 0.02 (0.01-0.07) 0.01 (0.00-0.04) 0.02 (0.01-0.07) 0.04 (0.01-0.17) 0.14 (0.03-0.45)
Median SOFA (IQR) 4 (2-6) 3 (1-4) 4 (3-6) 5 (4-7) 7 (5-10)
Admission Diagnosis




    Medical 197,553 (30%) 74,842 (24%) 51,037 (29%) 55,029 (44%) 16,645 (61%)
    Post-Operative 191,807 (30%) 112,503 (35%) 53,720 (30%) 22,809 (18%) 2,775 (10%)
    Sepsis 52,894 (8.2%) 23,625 (7.5%) 14,518 (8.2%) 11,587 (9.2%) 3,164 (12%)
    Trauma/Neurosurgery 77,311 (12%) 50,410 (16%) 18,202 (10%) 7,660 (6.1%) 1,039 (3.8%)
    Cardiac Surgery 128,286 (20%) 55,633 (18%) 40,283 (23%) 28,504 (23%) 3,866 (14%)
COVID Penumonitis (Proven) 4,865 (0.7%) 265 (<0.1%) 657 (0.4%) 2,569 (2.0%) 1,374 (4.9%)
Admission Source




    Emergency Department 158,625 (24%) 67,604 (21%) 41,263 (23%) 38,897 (31%) 10,861 (40%)
    Operating Theatre/Recovery 387,416 (59%) 214,294 (67%) 109,261 (61%) 56,536 (45%) 7,325 (27%)
    Ward 74,547 (11%) 25,911 (8.1%) 19,449 (11%) 22,067 (18%) 7,120 (26%)
    ICU, Same Hospital 658 (0.1%) 276 (<0.1%) 168 (<0.1%) 165 (0.1%) 49 (0.2%)
    Other Hospital 29,930 (4.6%) 12,646 (3.9%) 8,065 (4.5%) 7,396 (5.9%) 1,823 (6.7%)
    Direct from Home 675 (0.1%) 298 (<0.1%) 172 (<0.1%) 158 (0.1%) 47 (0.2%)
Hospital Type




    Tertiary 300,036 (45%) 141,908 (44%) 82,532 (45%) 62,179 (48%) 13,417 (48%)
    Metropolitan 94,431 (14%) 39,233 (12%) 25,487 (14%) 22,858 (18%) 6,853 (24%)
    Rural / Regional 62,039 (9.4%) 25,484 (7.8%) 17,587 (9.7%) 15,416 (12%) 3,552 (13%)
    Private 206,106 (31%) 118,136 (36%) 55,893 (31%) 27,774 (22%) 4,303 (15%)
Chronic Respiratory Disease 47,897 (7.2%) 14,304 (4.4%) 15,910 (8.8%) 14,645 (11%) 3,038 (11%)
Chronic CVS Disease 60,583 (9.1%) 26,453 (8.1%) 17,915 (9.9%) 13,388 (10%) 2,827 (10%)
Chronic Hepatic Disease 12,012 (1.8%) 5,202 (1.6%) 3,383 (1.9%) 2,749 (2.1%) 678 (2.4%)
Chronic Renal Disease 23,002 (3.5%) 10,724 (3.3%) 6,478 (3.6%) 4,720 (3.7%) 1,080 (3.8%)
Frailty




    Fit/Well 279,572 (59%) 147,891 (63%) 73,133 (57%) 48,604 (53%) 9,944 (51%)
    Mild 142,545 (30%) 64,841 (28%) 41,025 (32%) 30,223 (33%) 6,456 (33%)
    Moderate/Severe 50,901 (11%) 20,270 (8.7%) 15,069 (12%) 12,546 (14%) 3,016 (16%)
1 n; Median (Q1-Q3); n (%)

3.2 ICU Supports

Characteristic Overall1
Acute Respiratory Failure Category
None (PF >300)1 Mild (PF 200-300)1 Moderate (PF 100-200)1 Severe (PF < 100)1
IPPV on Day 1 245,116 (37%) 94,906 (29%) 69,017 (38%) 63,782 (50%) 17,411 (62%)
IPPV 254,129 (38%) 97,466 (30%) 71,640 (39%) 66,789 (52%) 18,234 (65%)
Hours of IPPV (Pts that received IPPV) 18 (8, 62) 15 (7, 41) 16 (7, 49) 24 (10, 89) 51 (16, 156)
NIV 69,929 (11%) 13,271 (4.1%) 20,584 (11%) 28,181 (22%) 7,893 (28%)
Hours of NIV (Pts that received NIV) 13 (5, 33) 9 (4, 23) 13 (5, 30) 15 (6, 38) 15 (5, 40)
ECMO 2,120 (0.3%) 295 (<0.1%) 268 (0.1%) 683 (0.5%) 874 (3.1%)
Inotropes/Vasopressors 266,049 (40%) 109,241 (34%) 74,969 (41%) 64,608 (50%) 17,231 (61%)
Tracheostomy 8,856 (1.3%) 2,894 (0.9%) 2,194 (1.2%) 2,684 (2.1%) 1,084 (3.9%)
Acute Kidney Injury 30,315 (4.6%) 10,503 (3.2%) 7,892 (4.3%) 8,427 (6.6%) 3,493 (12%)
Renal Replacement Therapy 27,885 (4.2%) 8,775 (2.7%) 6,738 (3.7%) 8,515 (6.6%) 3,857 (14%)
1 n (%); Median (Q1, Q3)

3.3 ICU & Hospital Outcomes

Characteristic Overall1
Acute Respiratory Failure Category
None (PF >300)1 Mild (PF 200-300)1 Moderate (PF 100-200)1 Severe (PF < 100)1
Hospital Mortality 57,052 (8.6%) 15,797 (4.9%) 14,291 (7.9%) 18,247 (14%) 8,717 (31%)
ICU Mortality 38,681 (5.9%) 9,138 (2.8%) 8,897 (4.9%) 13,212 (10%) 7,434 (26%)
Mean ICU Length of Stay in Days (SD) 3.33 (5.57) 2.60 (4.19) 3.23 (5.32) 4.64 (7.13) 6.35 (9.50)
Median ICU Length of Stay in Days (IQR) 1.89 (0.96-3.70) 1.63 (0.90-2.88) 1.92 (0.98-3.67) 2.79 (1.46-5.15) 3.48 (1.54-7.43)
Mean Hospital Length of Stay in Days (SD) 15 (86) 14 (87) 15 (84) 16 (88) 17 (76)
Median Hospital Length of Stay in Days (IQR) 9 (5-15) 8 (4-14) 9 (5-16) 10 (6-17) 10 (4-19)
1 n (%); Mean (SD); Median (Q1-Q3)

3.4 Mortality Outcomes

Below are the cumulative incidences of hospital mortality, ICU mortality, 6-month mortality and 12-month mortality.

Characteristic Overall1
Acute Hypoxaemic Respiratory Failure Category
None (PF >300)1 Mild (PF 200-300)1 Moderate (PF 100-200)1 Severe (PF < 100)1
Number of Patients 662,612 324,761 181,499 128,227 28,125
ICU Mortality 38,681 (5.9%) 9,138 (2.8%) 8,897 (4.9%) 13,212 (10%) 7,434 (26%)
Hospital Mortality 57,052 (8.6%) 15,797 (4.9%) 14,291 (7.9%) 18,247 (14%) 8,717 (31%)
28-Day Mortality 53,397 (8.1%) 15,715 (4.8%) 13,874 (7.6%) 16,882 (13%) 6,926 (25%)
90-Day Mortality 70,842 (11%) 23,570 (7.3%) 18,689 (10%) 20,734 (16%) 7,849 (28%)
180-Day Mortality 84,821 (13%) 30,530 (9.4%) 22,542 (12%) 23,380 (18%) 8,369 (30%)
1-Year Mortality 104,441 (16%) 40,498 (12%) 27,923 (15%) 27,008 (21%) 9,012 (32%)
1 n; n (%)

3.5 Primary Outcome: Hospital Mortality

3.6 Primary & Secondary Outcomes: Discriminator Characteristics

Criteria Hospital Mortality ICU Mortality 28-Day Mortality 90-Day Mortality 180-Day Mortality 1-Year Mortality
Optimal Cutpoint 230.000 224.286 233.333 233.333 233.333 233.333
Sensitivity (Sn) 0.561 0.602 0.546 0.502 0.471 0.441
Specificity (Sp) 0.706 0.719 0.694 0.696 0.696 0.697
Positive Predictive Value (PPV) 0.152 0.118 0.135 0.165 0.186 0.214
Negative Predictive Value (NPV) 0.945 0.967 0.946 0.921 0.900 0.869
Accuracy 0.694 0.712 0.682 0.675 0.668 0.656
Youden's Index 0.267 0.321 0.240 0.198 0.168 0.137
Area Under Curve (AUC) 0.677 0.709 0.659 0.632 0.612 0.591

3.7 Primary & Secondary Outcomes: Receiver Operator Curves

3.8 Subgroup Analysis

3.8.1 Intubated & Non-Intubated Patients: Discrimination Statistics

Discriminator Characteristics for PaO2:FiO2 Ratio
In Patients Receiving IPPV
Criteria Hospital Mortality ICU Mortality 28-Day Mortality 90-Day Mortality 180-Day Mortality 1-Year Mortality
Optimal Cutpoint 173.333 173.913 180.000 179.348 187.778 175.000
Sensitivity (Sn) 0.431 0.463 0.424 0.408 0.420 0.367
Specificity (Sp) 0.767 0.764 0.745 0.746 0.722 0.759
Positive Predictive Value (PPV) 0.221 0.189 0.183 0.207 0.215 0.242
Negative Predictive Value (NPV) 0.897 0.923 0.906 0.886 0.872 0.852
Accuracy 0.722 0.732 0.707 0.699 0.675 0.692
Youden's Index 0.197 0.227 0.169 0.154 0.141 0.127
Area Under Curve (AUC) 0.627 0.645 0.610 0.601 0.592 0.581

Discriminator Characteristics for PaO2:FiO2 Ratio
In Patients Not Receiving IPPV
Criteria Hospital Mortality ICU Mortality 28-Day Mortality 90-Day Mortality 180-Day Mortality 1-Year Mortality
Optimal Cutpoint 253.333 234.694 252.381 261.905 264.000 262.162
Sensitivity (Sn) 0.589 0.653 0.558 0.519 0.488 0.451
Specificity (Sp) 0.712 0.755 0.714 0.691 0.686 0.694
Positive Predictive Value (PPV) 0.107 0.068 0.108 0.142 0.171 0.212
Negative Predictive Value (NPV) 0.967 0.988 0.963 0.936 0.910 0.874
Accuracy 0.705 0.753 0.705 0.675 0.663 0.656
Youden's Index 0.300 0.408 0.272 0.210 0.174 0.144
Area Under Curve (AUC) 0.698 0.763 0.680 0.640 0.616 0.596

3.8.2 Intubated and Non-Intubated (Excluding Patients with Treatment Limitations)

Discriminator Characteristics for PaO2:FiO2 Ratio
In Patients Receiving IPPV with No Treatment Limitations
Criteria Hospital Mortality ICU Mortality 28-Day Mortality 90-Day Mortality 180-Day Mortality 1-Year Mortality
Optimal Cutpoint 177.778 179.000 180.000 180.000 187.778 177.500
Sensitivity (Sn) 0.454 0.496 0.437 0.418 0.426 0.380
Specificity (Sp) 0.756 0.750 0.748 0.749 0.725 0.754
Positive Predictive Value (PPV) 0.183 0.153 0.158 0.181 0.189 0.213
Negative Predictive Value (NPV) 0.920 0.942 0.925 0.906 0.894 0.874
Accuracy 0.724 0.729 0.717 0.710 0.686 0.698
Youden's Index 0.210 0.246 0.184 0.167 0.151 0.133
Area Under Curve (AUC) 0.637 0.659 0.622 0.611 0.600 0.587

Discriminator Characteristics for PaO2:FiO2 Ratio
In Patients Not Receiving IPPV with No Treatment Limitations
Criteria Hospital Mortality ICU Mortality 28-Day Mortality 90-Day Mortality 180-Day Mortality 1-Year Mortality
Optimal Cutpoint 260.714 231.034 266.667 260.000 260.000 267.273
Sensitivity (Sn) 0.557 0.615 0.550 0.460 0.422 0.411
Specificity (Sp) 0.705 0.781 0.688 0.709 0.710 0.692
Positive Predictive Value (PPV) 0.057 0.033 0.059 0.090 0.116 0.150
Negative Predictive Value (NPV) 0.980 0.994 0.977 0.954 0.932 0.899
Accuracy 0.700 0.779 0.683 0.695 0.686 0.659
Youden's Index 0.262 0.396 0.238 0.170 0.132 0.103
Area Under Curve (AUC) 0.673 0.757 0.658 0.612 0.588 0.569

3.8.3 A Priori Subgroups

Comparison of AUCs of Each Subgroup
Subgroup Comparison AUC 1 Vs AUC 2 p-value
Day 1 IPPV Yes Vs No 0.628 Vs 0.701 <0.001
IPPV Any Time Yes Vs No 0.627 Vs 0.698 <0.001
NIV Any Time Yes Vs No 0.622 Vs 0.668 <0.001
ECMO Any Time Yes Vs No 0.516 Vs 0.675 <0.001
Admission Diagnosis Cardiac Vs NeuroSurgery 0.62 Vs 0.641 0.008
Cardiac Vs Post-Operative 0.62 Vs 0.647 <0.001
Cardiac Vs Medical 0.62 Vs 0.644 <0.001
Cardiac Vs Sepsis 0.62 Vs 0.661 <0.001
Neurosurgery Vs Post-Operative 0.641 Vs 0.647 0.326
Neurosurgery Vs Medical 0.641 Vs 0.644 0.587
Neurosurgery Vs Sepsis 0.641 Vs 0.661 0.001
Post-Operative Vs Medical 0.647 Vs 0.644 0.422
Post-Operative Vs Sepsis 0.647 Vs 0.661 0.006
Medical Vs Sepsis 0.644 Vs 0.661 <0.001
Age Categories Age <44 Vs Age 45-64 0.704 Vs 0.675 <0.001
Age <44 Vs Age 65-84 0.704 Vs 0.67 <0.001
Age <44 Vs Age >84 0.704 Vs 0.67 <0.001
Age 45-64 Vs Age 65-84 0.675 Vs 0.67 0.116
Age 45-64 Vs Age >84 0.675 Vs 0.67 0.309
Age 65-84 Vs Age >84 0.67 Vs 0.67 0.96
Sex Male Vs Female 0.671 Vs 0.684 <0.001
Frailty Fit/Well Vs Mild Frailty 0.687 Vs 0.667 <0.001
Fit/Well Vs Moderate/Severe Frailty 0.687 Vs 0.638 <0.001
Mild Frailty Vs Moderate/Severe Frailty 0.667 Vs 0.638 <0.001
Treatment Limitations Yes Vs No 0.624 Vs 0.68 <0.001

4 References

1.
Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32-35. doi:10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3
2.
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837-845.
3.
Fairbanks M. tidytable: Tidy Interface to ’data.table’. doi:10.32614/CRAN.package.tidytable
4.
Wickham H. tidyverse: Easily Install and Load the ’Tidyverse’. doi:10.32614/CRAN.package.tidyverse
5.
Barrett T, Dowle M, Srinivasan A, et al. data.table: Extension of ‘data.frame‘. doi:10.32614/CRAN.package.data.table
6.
Sjoberg DD, Curry M, Hannum M, Larmarange J, Whiting K, Zabor EC. Gtsummary: Presentation-Ready Data Summary and Analytic Result Tables.; 2021. https://CRAN.R-project.org/package=gtsummary.
7.
Iannone R, Cheng J, Schloerke B, Hughes E. Gt: Easily Create Presentation-Ready Display Tables.; 2022. https://gt.rstudio.com/authors.html#authors.
8.
Thiele C. cutpointr: Determine and Evaluate Optimal Cutpoints in Binary Classification Tasks. doi:10.32614/CRAN.package.cutpointr
9.
Robin X, Turck N, Hainard A, et al. pROC: Display and Analyze ROC Curves. doi:10.32614/CRAN.package.pROC